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ABSTRACT 



Decision verification in quality assurance is simply the Government’s 
determination of whether or not a contractor's inspection decisions are 
correct. This paper discusses procedures for accomplishing decision 
verification both as prescribed by directives and as actually done. It 
examines reasons why decision verification does not appear to be a viable 
element in current quality assurance programs even though this procedure 
is still prescribed in Navy directives for use by contract administrators. 
The use of operating characteristic curves as a managerial aid to deter- 
mine whether or not to use decision verification is also discussed. 
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I. INTRODUCTION 



Because of the shift in emphasis away fron "hands-on" inspection 
by the Government in quality assurance, effective methods of evaluating 
the contractor’s quality program have been sought by government agen- 
cies responsible for contract administration. Decision verification is 
one procedure which was developed to assist in assessing the effective- 
ness of a contractor in controlling product quality. 

Decision verification involves the Government's determination as to 
whether contractor inspection decisions are correct or incorrect. 
Statistical methods are then used to determine if the contractor's in- 
spection process is effective under criteria established by the Govern- 
ment. 

Decision verification has been applied with varying degrees of in- 
terest, enthusiasm, and success. Some of the Government agencies which 
adopted these procedures in the 1960 's have since ceased to use them 
because of real or perceived difficulties. In this paper we shall be 
concerned with what the problems are, and how to overcome them so that 
decision verification can be a useful means of judging the inspection 
decision process of the contractor. The basic approach is to: 

1. Discuss how decision verification should be accomplished if 
done in accordance with guidance contained in Navy directives. 

2. Discuss how decision verification was observed to be done at 
selected activities, if it was done at all. 

3. Show how the specified decision verification procedures were 
developed, and suggest what could be done to improve the acconplishment 
of decision verification. 
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As an initial step in research, applicable directives, reports, 
and other documents dealing with decision verification were examined. 
This provided a basic knowledge of how decision verification should 
be done, at least in theory. Next, visits to several production 
facilities and interviews with quality assurance personnel at these 
facilities and in other Government offices provided information on hew 
decision verification really was done and the problems encountered in 
doing it. 

The plan of this paper is to compare the specified with the actual. 
Early discussion centers on the quality assurance policy of the Depart- 
ment of Defense, which caused procedures like decision verification to 
be developed. Then the relationship of decision verification to other 
quality assurance procedures in the overall Navy quality assurance pro- 
gram is discussed. How various decision verification procedures differ 
within the Navy is also highlighted. This is followed by a discussion 
of the "real” world procedures or how decision verification is actually 
done in the field. The final section of this paper contains conclu- 
sions and recommendations. 

Now, to set the scene, the next chapter discusses Department of 
Defense quality assurance policy. 
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II. DECISION VERIFICATION IN DOD 
QUALITY ASSURANCE PROGRAMS 



Decision verification procedures are used to determine whether or 
not a contractor's inspection decisions are correct or incorrect, and 
with this information in hand, to assess the effectiveness of the con- 
tractor's inspection process. This chapter deals with the application 
of decision verification procedures by the Navy and other organizations 
within the Department of Defense. The purpose of this chapter is to 
tell why these procedures were developed, and where they fit into the 
quality assurance programs of various organizations within the Depart- 
ment of Defense. This chapter gives some reasons for the emphasis on 
quality assurance by Government procurement organizations and discusses 
DOD quality assurance policy. It outlines the basic elements of quality 
assurance programs of two U. S. Navy organizations and highlights some 
of the differences which exist in these programs. 

A. IMPORTANCE OF QUALITY ASSURANCE 

Quality assurance is an important part of systems procurement by 
the Department of Defense. It is easy to cite at least three reasons 
for this. 

First, many of the current systems purchased by DOD are highly com- 
plex. This is true not only for major systems such as aircraft and 
ships, but also for smaller weapons systems. For example, the MK 56 
sea mine requires 140 individual components for assembly of the completed 
weapon. Effective quality assurance improves the probability that these 
140 components will be assembled into a unit that will work as designed. 
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The second reason is closely allied to the first. Frequently, 
many of- the individual components needed for the complete system are 
produced by different manufacturers using a variety of processes. 

Many prime and sub-contractors produce items which must function 
together in the end product. For example, over 125 different contrac- 
tors are manufacturing hardware for the NASA Space Shuttle. These com- 
ponents and subassemblies range fran simple transformers to such 
complex equipment as the Space Shuttle's main engines [Ref. 1]. Effec- 
tive quality assurance is needed in order to minimize the problems 
associated with assembling all of these different components into one 
major vehicle. 

A third reason for the emphasis on quality assurance is the avoid- 
ance of financial waste associated with systems which fail to perform 
as designed. A communications or intelligence satellite program, 
which can cost well over $100 million, may waste substantial resources 
in support equipment and facilities if the space vehicle does not 
survive in orbit as long as intended. 

B. DOD QUALITY ASSURANCE PHILOSOPHY AND POLICY 

The current EOD philosophy on quality assurance was initially de- 
veloped in 1954 and formally expressed in DODINST 4155.6. This instruc- 
tion made contractors responsible for assuring that material submitted 
for acceptance to the Government did meet contract specifications and 
requirements. The contractor had always been responsible for producing 
material to the requirements of the contract, but prior to 1954 it had 
been up to the Government to determine whether or not the material was 
acceptable. Since acceptability was determined by inspection of the 
material itself, a considerable amount of "hands-on" inspection by the 
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Government was necessary. Beginning in 1954 there was a redirection 
of quality assurance emphasis away frcm product inspection and toward 
the assessment of the contractor's quality assurance program. The 
Department of Defense no longer performed the extensive product in- 
spections which had formerly characterized its approach to quality 
assurance. Instead, the contractor was now responsible for performing 
many of the product inspections previously done by the Government 
quality assurance representative [Ref. 2] . 

The basic DOD procurement quality assurance policy can be found in 
Armed Services Procurement Regulations [Ref. 3] . This document sets 
forth the quality assurance responsibilities of the Government and of 
the contractor, and issues policy guidance for quality assurance rather 
than detailed plans for establishing and operating an effective quality 
program. 

These regulations make it clear that the Government must determine 
the quality requirements which are to be included in the contract. 

This is usually done by contractual provisions dealing with performance 
specifications, product specifications, standard inspection clauses, 
and other quality assurance related specifications such as MTL-Q-9858A, 
Quality Assurance Program Requirements . 

The contractor is assigned responsibility for controlling the . 
quality of his product and for assuring that only contractually conform- 
ing items are delivered to the Government. He must also maintain, and 
furnish to the Government if requested, objective evidence that he is 
fulfilling his quality responsibilities. 

Armed Services Procurement Regulations do not prescribe specific 
quality-assurance measures. In accordance with these regulations: 
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It is the contract administration office responsibility to 
develop and apply effective and efficient procedures for 
Government procurement quality assurance. 

The regulations list a number of possible quality assurance procedures 

from which the responsible contract administration office may choose 

for administering the quality requirements of a given contract. These 

options include, but are not limited to the following: 

1. Physical inspection of the contract item. 

2. Evaluation and approval or disapproval of the contractor's 
system for controlling quality. 

3. Use of information frcm consumers to uncover deficiencies. 

Now that Defense Department quality assurance policy has been out- 
lined, we will discuss some of the organi 2 ations that carry out that 
policy, with particular emphasis on organizations that administer Navy 
contracts . 

Navy contracts are administered by one of several sources. Some 
of these contracts are administered by the office of a Naval Plant 
Representative which is located at the contractor's plant. Others are 
administered by the office of the Supervisor of Shipbuilding, Conver- 
sion, and Repair located at each Naval District headquarters. Navy con- 
tracts for material which is being produced in plants under the primary 
cognizance of another Service such as the Air Force are normally 
administered by that Service. The Defense Contract Administration 
Service often administers contracts for military material production 
in plants where no Service personnel are assigned. 



1 U. S. Department of Defense, Armed Services Procurement Regulations , 
(Washington: 1 July 1976), p. 14:8. 
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It is interesting to note that Defense Department agencies includ- 
ing the Navy are sometimes tasked to administer contracts for other 
Government organizations outside of the Department of Defense. For ex- 
ample, the Defense Contract Administration Service has been delegated 
to act for NASA on seme Space Shuttle contracts. NASA does not prescribe 
all of the detailed quality assurance actions which the delegated agency 
should take. The Defense Contract Administration Service must use its 
existing procedures as well as any guidance or special instructions which 
NASA provides [Ref. 4] . 

C. SOME DIFFERENCES IN QUALITY ASSURANCE 

PROGRAMS OF VARIOUS AGENCIES 

It has been pointed out that Navy contracts are sometimes adminis- 
tered by other agencies, and that in turn the Navy may be called upon 
to administer other than Navy contracts. Since the contract administra- 
tion activity generally uses its own quality assurance procedures on 
all contracts that it administers, it is important to note seme of the 
differences that exist between the quality assurance programs of the 
various DOD agencies. 

Requirements for DOD quality assurance programs are contained in 
Defense Supply Agency Manual 8200.1 of August 1976 which is entitled, 
Procurement Quality Assurance [Ref. 5] . Although this is a Joint Ser- 
vice directive and was co-signed by representatives of each military 
Service, the Navy has not yet chosen to abide by this manual without 
issuing its own supplemental instructions. 

In connection with this discussion of decision verification, it is 
interesting to note that one of the predecessors of the present manual 
was issued in 1969. This directive, commonly referred to as Handbook 
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57, contained a decision verification procedure called "Contractor 
Decision Verification" [Ref. 6] . The requirement to accomplish Contrac- 
tor Decision Verification was cancelled in 1973 when Handbook 57 was 
superseded by an early version of Defense Supply Agency Manual 8200.1 . 
Consequently, the Defense Contract Administration Service, which organi- 
zationally is part of the Defense Supply Agency, no longer uses 
Contractor Decision Verification procedures in administering contracts. 
Thus we see that a very important part of the DOD contract administra- 
tion organization omits decision verification as an element in quality 
assurance. 

Unlike the Defense Contract Administration Service, the Navy has 
not eliminated decision verification from its repertory of quality- 
assurance procedures. At least two organizations within the Navy 
material headquarters have issued quality assurance instructions con- 
taining decision verification procedures to be used by their field 
contract-administration offices . 

The Naval Air Systems Command is one of the two organizations . 

It exercises control over contract administration of production in 
some contractors’ plants through the office of a Naval Plant Represen- 
tative or NAVPRO who is physically located at the production facility. 

He is responsible for developing and applying effective and efficient 
quality assurance plans and procedures which are consistent with 
requirements of the particular contracts which he administers. 

The basic elements of the quality assurance program for use by 
contract-administration field activities which are under the cognizance 
of the Naval Air Systems Command are as follows: 
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